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ABSTRACT 

Maximum  likelihood  estimates  for  errors-in-variables  models  are  not 
always  root-N  consistent.  We  provide  an  example  of  this  for  logistic 
regression. 


SOME  KEY  WORDS:  Binary  regression.  Measurement  error.  Logistic  regression. 
Maximum  likelihood,  Functional  models. 


I .  INTRODUCTION 


Logistic  regression  is  a  popular  device  for  estimating  the  probability 
of  an  event  such  as  the  development  of  heart  disease  from  a  set  of  predictors, 
e.g.,  systolic  blood  pressure.  The  simplest  form  of  this  model  is  logistic 
regression  through  the  origin  with  a  single  predictor: 


Cl)  Pr{Y.=l|c.}  =  G(a0c.)  =  (l+expC-^c.)  }-1  , 

where  aQ  and  (c^  are  scalars  (i=l,...,N).  In  many  applications,  the  predictors 
are  measured  with  substantial  error;  a  good  example  of  this  is  systolic  blood 
pressure,  see  Carroll,  et  al  (1983).  Thus,  we  observe 
(2)  C±  =  c±  *  vt  , 

where  the  errors  {v^  are  assumed  here  to  be  normally  distributed  with  mean  zero 
2 

and  variance  o  . 

The  functional  errors-in-variables  logistic  regression  model  is  the  case 
where  (1)  and  (2)  hold  and  the  true  values  {c^}  are  unknown  constants.  The 
parameters  are  ctQ  and  {c^};  there  are  (N+l)  parameters  with  N  observations,  so 
classical  maximum  likelihood  theory  does  not  apply.  Up  to  a  constant,  the 
log-likelihood  is 


-N  log  a  -  (2a2)'1  l  CC.-c.)2 
e  i*l  1  1 


N 

♦  l  (Y.  log  G(ac.)+(l-Y.)log  (1-G(ac.))-)  . 
i«l  1  e  i  l  e  i 

The  linear  functional  errors  in  variables  model  (Kendall  and  Stuart  (1979)) 

takes  a  form  similar  to  (1)  and  (2),  although  of  course  (1)  is  replaced  by  the 

2  2  2  2  2 

usual  linear  regression  model  with  variance  o  .  If  o  ,  o  or  o  /o  is  known. 


then  the  linear  functional  maximum  likelihood  estimate  exists  and  is  both 

consistent  and  asymptotically  normally  distributed. 

In  this  note,  we  show  that  for  the  functional  logistic  errors-in-variables 

2 

model  (1)  and  (2),  even  if  a  is  known,  the  maximum  likelihood  estimate  cannot 

be  consistent  and  asymptotically  normally  distributed  about  aQ.  The  result 

can  be  extended  to  multiple  logistic  regression,  and  it  is  true  even  it  we 

replicate  (2)  a  finite  number  M  times.  If  the  number  of  replicates  M  -*•  00  as 

the  sample  size  N  then  the  functional  maximum  likelihood  estimate  can 

2 

be  shown  to  be  consistent  whether  a  is  known  or  not. 

II.  THE  THEOREM 

In  model  (1)  with  the  {ci}  known,  the  ordinary  maximum  likelihood  estimate 

for  an  satisfies 

u  N 

0  =  l  c  (Y.-GCO.C  ))  . 
i=l  1  1  11 

In  the  presence  of  measurement  error,  the  naive  estimator  would  solve 

N 

(3)  0  =  l  C  (Y.-G(lC.))  . 

i=l  1  1  *  1 

However,  because  of  correlations,  it  turns  out  that 

i  N 

(4)  lim  EN'1  l  C.(Y.-G(anC.))  4  0  . 

N-*»  i-1  u 

Condition  (4)  says  that  the  defining  equation  (3)  for  a 2  is  not  even 
consistent  at  the  true  value  a^.  Under  these  circumstances,  it  is  well 
known  from  the  theory  of  M-estimators  that  the  usual  naive  estimator  a2 
converges  not  to  otg  but  to  the  value  a*  satisfying 

i  N 

lim  E  N'1  l  C.  (Y.-G(cx*C.))  =  0  , 
i=l  1 

assuming  such  a  value  aA  exists  and  is  unique. 


Assuming  it  exists  and  is  unique,  the  functional  MLE  aQ  satisfies  an 
equation  analogous  to  (3) : 

i  N 

(s)  o  =  n'1  , 

where 

(6)  c.(a)  =  C.  +  aa2CY.-G(aci(a)))  . 

It  is  easy  to  construct  examples  for  which  an  analogue  to  (4)  holds: 

i  N 

(7)  lim  E  N"1  l  a  (a 0)(Y.-G(a  8 .(a )))  t  0. 

i=l  1  u  u  1  u 

One  example  of  (7)  is  the  extraordinarily  easy  problem  o2  =  1  and  c^  =  (-1)*. 

The  only  question  is  whether  (7)  is  enough  to  guarantee  that  the  functional 
MLE  aQ  cannot  be  asymptotically  normally  distributed  about  the  true  value  aQ. 

This  is  the  case. 

2 

Theorem  Suppose  that  o  is  known  and  that 

(A. 1)  The  maximum  likelihood  estimate  exists; 

i  N 

(A. 2)  N‘a  l  c.  -  A  (|A|  <  »)  ; 

i=l 

i  N  ? 

(A. 3)  N-1  l  c.  -*•  B  (0  <  B  <  »)  . 

i*l  1 

Then,  if 

(A. 4)  N*(a0-a0)  =  0p( 1)  , 

we  must  have  that  (7)  fails,  i.e., 
i  N 

(8)  lim  E  N"1  l  6.(a0)(Y  -G(a08.(a  )))  =  0. 

i-1  1  u  1  u  1  u 

Hie  theorem  as  stated  does  not  readily  follow  from  the  theory  of  M-estimators 
unless  one  assumes  the  existence  of  a  unique  ot*  which  satisfies  (8),  along  with 
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other  regularity  conditions.  The  proof  we  give  avoids  these  complications 
because  it  exploits  the  form  of  the  logistic  function  G. 


III.  PROOF  OF  TOE  THEOREM 

2  A 

It  is  most  transparent  to  take  a  =1.  By  formal  differentiation,  ot^ 

simultaneously  satisfies  (6)  and 


(9)  N-1  l  6.(a){G(o6.Ca))  -  Y  }  =  0. 

i=l 

Assumptions  (A. 2)  and  (A. 3)  imply  that 


max{cf/N  :  1  s  i  s  N}  -*■  0 


From  (2)  and  (6),  it  follows  that 


(11)  lim  max  sup  |c  (a)-v. |/(l+|a  |+|c. |)  =  0(1) 

c+0  UisN  |a-aQ|<e  11  u  i  p 

Further,  since  the  (v^  are  normally  distributed, 

1  P 

(12)  maxdvjN"*  :  1  S  i  S  N}  +  0  . 

Lemma  It  follows  that  if  (A. 1)-(A.4)  hold,  then 

(13)  max{|ci(aQ)  -  ciCaQ)|:  1  s  i  £  N)  S  0. 

Proof  of  the  Lemma.  Define 

H..(u,a)  ■  u-c^-v. +a{G(ou)  *  Y^}  , 

Hi(ci(a),  a)  *  0. 

The  partial  derivatives  of  are 
DjH^UjO)  «  ■  1  ■  a2G(au){l-G(au)}  , 

DjH^C^a)  «  ^HjCu.a)  »  (G(au)  -  ♦  auG(au) (I-G(au) }  . 

By  the  chain  rule, 


'iW'  i'.',',,'  /.  ■  V  V  V  .  .  V  -.*  •/  «"  - 


(14) 


From  (10) - (12)  and  (14)  it  follows  that  for  every  M  >  0, 


max  sup  . 
l£i£N  |  a-cig  |  <M/N~ 


=  0  {  max  sup  ,  I c.  (ot)  J /N*5} — — ►  0 

p  l*i*N  |a-aQ| <M/N~ 

This  means  that  for  every  M  >  0, 

max  sup  .  | 6. (a)  -  6 . (a.) |  -£— ►  0  , 

l<i<N  |a-a0|<M/N^ 

which  by  (A. 4)  completes  the  proof  of  the  Lemma. 


□ 


We  must  prove  that  (A.1)-(A.4)  imply  (8).  We  are  first  going  to  show 

that 

N  p 

(15)  N’1  ^c.(a0){G(a0c.(a0))-Yi}  -  0. 

The  term  in  (15)  can  be  written  as  AJN  +  A2N  +  A3N,  where 


Al„  ■  -  V 

A2N  =  ^  ^a0^i^a0^~  G^0^i  ^0^  ^  ’ 

*3n  •  -  yi]  • 


By  (9),  Aw  =  0  and,  since  G  is  bounded,  the  Lemma  and  (A. 4)  gives  A 

Jli 

Because  G  and  its  derivative  are  bounded,  the  Lemma  says  that  A 


IN 


2N 


as  long  as  N 


N'1  l  (8,(6L))2-  0  (1)  and  N'1  J  (6  (o0)>2  .  0  (1), 
i«l  1  u  p  i*l  v 


-6- 


which  follow  from  (A.3),  (10)  and  (11).  Since  (15)  holds,  to  prove  (8)  we 
aerely  need  to  show  that 

N'li  F  si(“oKvG(vi(v» 

1=1 

-E{a.(a0)(Y.-G(a06i(a0)))} 

This  follows  from  Chebychev’s  inequality  and  (A.3),  completing  the  proof.  □ 
The  Theorem  does  not  follow  from  ordinary  likelihood  calculations  because 
the  number  of  parameters  increases  with  the  sample  size. 

IV.  A  SIMULATION  STUDY 

To  give  some  idea  of  the  effect  of  measurement  error,  we  conducted  a 
small  Monte-Carlo  study  of  the  logistic  regression  model 

PHYj  =  1}  =  G(Ci/2  -  1),  i«l,...,N 

Here  the  values  {c^  were  randomly  generated  as  normal  random  variables  with 

2 

mean  zero  and  variance  3  =  ac>  while  the  measurement  errors  were  normally 

2 

distributed  with  mean  zero  and  variance  2  =  ay,  with  each  {c^}  being 
replicated  twice.  We  chose  the  two  sample  sizes  N  =  200,400  and  took  100 
simulations  for  each  sample  size. 

In  Table  1  we  report  the  Monte-Carlo  efficiencies  of  the  usual  naive 
estimator  and  the  functional  MLE  with  respect  to  the  logistic 
regression  based  on  the  correct  values  {c^}.  If  the  replicates  of  c^  are 
^il’Ci2*  we  use<*  S.  *  (Cii+Ci2)/2  and  estimated  the  variance  of  C^-c..  by  the 
sample  variance  of  (C^-  Ci2>/2' 

The  results  make  it  clear  that  neither  the  usual  naive  method  nor  the 
functional  MLE  are  acceptable.  Further  work  is  clearly  needed  to  identify 
good  methods. 


iWN’.V'.'.'i*" .**>■' /V!>  V  V 


TABLE  1 

Monte-Carlo  Mean  Squared  Error  Efficiencies 
Relative  to  Logistic  Regression  Based  On  The 
True  Predictors 

Pr{Yi=l|ci>  =  Gfoi+Bc^,  B  =  i,  a  =  -1.0 
i=l,...,N 

USUAL  FUNCTIONAL 

LOGISTIC  MLE 


a 

N=200 

0.74 

0.25 

N=400 

0.46 

0.32 

3 

N=200 

0.27 

0.15 

N=400 

0.09 

0.24 

a+B 

N=200 

1.13 

0.59 

N=400 

0.60 

0.53 

a+2B 

N=200 

0.38 

0.28 

N=400 

0.13 

0.43 
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